Penn State University Builds Learning Analytics Tool Through Instructure & AWS Solutions

A new learning analytics tool built by Penn State University is helping instructors track student engagement, offer differentiated student care, and enhance course delivery and learning outcomes. Utilizing data from Canvas learning management system (LMS), the tool – Course Insights – gives real-time alerts about how students engage within a course. 

Through Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Compute Cloud (Amazon EC2), Course Insights processes nearly 2 TB of raw learning activity data daily that can be embedded into Canvas LMS as a learning  tools interoperability (LTI) feature. To arrive at this solution, Penn State leveraged Amazon Web Services (AWS) cloud services and AWS Partner, Instructure.  

The Challenge: Retrieving Immediate Insights

Penn State first adopted Canvas in 2016, which played a critical role when the university needed to quickly adapt its LMS and other services for teaching and learning throughout the COVID-19 pandemic. Having since grown to serving roughly 89,000 students across 24 campuses in Pennsylvania, Penn State needed a solution for identifying students who may have been unable to transition to remote learning. 

The Data Empowered Learning team at Penn State first developed a tool to provide academic advisers with this information through an aggregated view of Canvas activity as a seven-day average for an entire class. While this gave instructors a bit of insight into whether students engaged in a course at the same rates or times as other students, there was still ambiguity around the specific actions students were taking.

Advisers and instructors were lacking an easy way to know if one of their students was not engaging with coursework, as well as a way to intervene in time to prevent that student from falling behind.

“Our faculty is hungry for data to support  students,” said Ben Hellar, manager of Data Empowered Learning at Penn State. “But most of the analytics that they had access to were post-course analytics. So how do we understand which students need help and intervention now? This motivation helped us design tools that deliver real-time data insight during the  semester so that instructors and academic advisers know when interventions can be most impactful.” 

Finding the Solution Through Faculty Feedback

Seeking more context, the Data Empowered Learning team asked faculty about their needs related to student support, course delivery, and pedagogy, all with the goal of understanding what information they find most useful about the courses they teach. The team found that what instructors wanted most was demographic data related to their students’ previously enrolled courses to better gauge their level of understanding at the start of the course. Also important, would be the ability to access this data in Canvas, where instructors said they spend most of their time. 

“We received feedback from instructors that told us, ‘I don’t want to use another tool—I live in Canvas—that is where I teach. So if you’re going to give me any new analytic data, put it in Canvas,’” Hellar said. 

Working with Instructure, Penn State built Course Insights, rolling it out Fall 2021 to a limited number of faculty members, and that number has since grown. “The nice thing about working with Instructure is we can share information and talk about what we’re building and why we’re building it,” said Andy Fisher, Manager of Learning  Engagement at Penn State. 

“Our faculty is hungry for data to support students,” Hellar said. “This motivation helps us  deliver real-time data insight during the semester so that instructors know when interventions can be most impactful.” 

The Results: An Invention that Impacts

Penn State launched Course Insights as a limited-access pilot and has since rolled it out to over 250 instructors in more than 4,000 courses. The tool pulls and analyzes data every day from Canvas and other learning tools like Kaltura, Paperback, and TopHat, as well as data from Starfish and student data found on the university's Student Information System (SIS).

Through AWS’s secure and resizable compute capacity for virtually any workload, Course Insights allows instructors to see a comprehensive view of student learning activity. Anomalous activity is highlighted and color-coded to make it user-friendly for instructors trying to pinpoint students in need of immediate support. The Data Empowered Learning Team of Penn State Information Technology received a Platinum 2023 Learning Impact Award from the 1EdTech Consortium for Course Insights. 

“This award helps validate that our approach of delivering learning analytic  data to instructors has value not only to the institution but for the industry at large,” Hellar said. 

The work Penn State has done with Course Insights has also influenced Instructure’s approach to Canvas data, documentation, standards and APIs – using the learnings to continue to inform its product roadmap. For example, Instructure updated Canvas to reduce the application’s latency, now reporting data closer to real time.

Taking the Tool Further

Penn State’s goal is to continuously improve Course Insights, enabling the tool to draw out more data insights such as analytics to help faculty and administrators at the department, college, and campus levels of the institution. The team also hopes to better understand course structure and student engagement with course material at scale, as that will bolster support for strategic initiatives related to retention and closing equity gaps. Through its close partnership with Instructure, Penn State can share learnings that may be useful for other institutions using Canvas. 

“We want all students to learn,” said Andy Fisher, manager of learning engagement at Penn State. “The nice thing about working with Instructure is we can share information and talk about what we’re building and why we’re building it. This is different  with education. We need to invent and we need to build what’s next. Some of this is very custom. But we always have that  conversation around, if we build something, can another university use it?”